Pre-Processing Device and Method Before Encoding of a Video Image Sequence

ABSTRACT

The invention relates to a method of processing an image of a video image sequence, wherein it comprises the following successive steps:
         a step for computing a complexity value representative of the complexity of said image;   a first step of morphological processing applied on said image, said first step generating a first processed image;   a second step for mixing said image and said first processed image depending on said complexity value, said second step generating a mixed image;   a third step of morphological processing applied on said mixed image, said third step generating a second processed image; and   a fourth step for mixing said mixed image and said second processed image depending on said complexity value.

The invention relates to a pre-processing device and method beforeencoding of a video image sequence.

The image encoding devices become all the more effective as the temporalor spatial entropy of the images they encode reduces.

They are therefore often associated with image pre-processing devices inwhich the images are processed in order to allow a better encoding.

As is known, the pre-processing devices for reducing the entropy of avideo sequence use linear or non-linear filters which reduce, eveneliminate, the high-frequency components that are mainly responsible forthe image encoding cost in intra mode. There are numerous filtersavailable, including one- or two-dimensional low-pass filters, Nagaofilters, averaging filters and median filters.

The main drawbacks with these methods are:

-   -   a reduction in spatial definition that is too visible, in        particular in the vertical axis, due to the fact that each frame        of an interlaced video has only half the vertical resolution of        an image,    -   blurring effects on the objects,    -   degraded contours.

The invention proposes to resolve at least one of the abovementioneddrawbacks.

To this end, the invention proposes a pre-processing device beforeencoding of a video image sequence, characterized in that it comprises:

-   -   means of applying a plurality of morphological processing steps        to the video image sequence,    -   mixers for applying a weighting, after each morphological        processing step, to the video sequence having been subjected to        one of said processing steps.

Applying a raw morphological processing would have a devastating effecton the quality of the image. The presence of a mixer between eachmorphological operator weights the effect of this processing by mixingthe raw result of the operator and the input of the same operator.

According to a preferred embodiment, the device comprises means ofmeasuring the complexity of said video image sequence before applyingthe plurality of morphological processing steps.

In practice, a pre-processing for reducing the spatial entropy of theimage is recommended mainly for images with high complexity. Thus thepre-processing can be controlled according to the complexity of theimage and suited to the complexity of the image.

Advantageously, the means of measuring the complexity of said videoimage sequence measure the intra-image correlation.

According to a preferred embodiment, the means of measuring thecomplexity compute a weighting coefficient for each mixer.

Advantageously, the weighting coefficient is identical for each mixer.

Advantageously, the weighting coefficients are inversely proportional tothe intra-image correlation.

Preferably, the means of applying a plurality of morphologicalprocessing steps and the mixers apply the processing steps to theluminance component of the video signal, pixel by pixel, for each image.

Preferably, the device comprises:

-   -   means of deinterlacing said video image sequence before        measuring the intra-image correlation and    -   means of interlacing said video sequence after the last        weighting.

This makes it possible to obtain progressive frames which each containthe complete vertical definition of an image. It is then possible toconsider without bias a processing in both axes of the image, horizontaland vertical.

The invention also relates to a method of pre-processing before encodinga video image sequence. According to the invention, the methodcomprises:

-   -   a plurality of morphological processing steps on the incoming        video image sequence,    -   a plurality of weighting steps for applying, after each        morphological processing step, a weighting to the result of the        morphological processing.

The invention will be better understood and illustrated by means ofexamples of embodiments and advantageous implementations, by no meanslimiting, with reference to the appended figures in which:

FIG. 1 represents a preferred embodiment of a device according to theinvention,

FIG. 2 represents the vicinity of the current point P taken into accountto define the complexity of the current image.

The modules represented are functional units, which may or may notcorrespond to physically distinguishable units. For example, thesemodules, or some of them, may be grouped in a single component, or formfunctionalities of one and the same software. Conversely, certainmodules may, if necessary, comprise separate physical entities.

The video signal Ei at the input of the pre-encoding device is aninterlaced type video signal.

In order to improve the performance of the pre-encoding device, thevideo signal Ei is deinterlaced by the deinterlacer 1. The deinterlacer1 doubles the number of lines per field of the video signal Ei using adeinterlacing method known to a person skilled in the art based on threeconsecutive fields of the video signal Ei. Progressive format is thenobtained, where each field is becoming frame and contains the completevertical definition of an image, so that a processing can be performedin the vertical axis.

The signal E is obtained at the output of the deinterlacer 1.

A complexity analysis of the image is then carried out. In practice,spatial entropy reduction is applied mainly to images having a highspatial entropy.

The device therefore includes upstream means 2 of measuring thecorrelation of each image.

FIG. 2 illustrates an example of the vicinity taken into account forcomputing the complexity of the current image in the module 2.

For each pixel of the image, the pixel result “Rp” is computed using theluminance of the current point and that of four of its neighbours:

Rp=[abs(P−P(−1,0))+abs(P−P(0,−1))+abs(P−P(0,+1))+abs(P−P(+1,0))]/4

Then, all these pixel results are accumulated over one frame.

$C_{intra} = \frac{\sum\limits_{0}^{nbpixels}{Rp}}{{nblignes} \times {nbcol}}$

This correlation measurement is used to ascertain the average deviation,over one image, between a pixel and its adjacent pixels. Interestinginformation on the definition in the image is thus obtained.

In other embodiments, it is possible to modify these equations in orderto obtain a more complete definition of the complexity of the image. Itis also possible to enlarge the vicinity of the current pixel taken intoaccount in computing the image complexity.

From this measurement, a coefficient K is computed within the range[0,1], as a function of the complexity of the incoming images.

The table below illustrates the values of the coefficient K as afunction of C_(intra), given as an illustration only.

The value of Cintra is encoded on 8 pixels and is therefore between 0and 255.

Correlation type C_(intra) value Coefficient K value Very strongC_(intra) = [0 . . . 2] 1/8 = Kmin Strong C_(intra) = [2 . . . 4] 2/8Average C_(intra) = [5 . . . 8] 3/8 Weak C_(intra) = [9 . . . 16] 4/8Very weak C_(intra) = [17 . . . 30] 5/8 Insignificant C_(intra) = [30 .. . 255] 6/8 = Kmax

When the correlation is very strong (very little definition), thepre-processing of the image is still performed, but in lesserproportions, illustrated by the value of the coefficient K.

Conversely, a weak correlation is an indicator of strong entropy. Theexample of a random noise (weak, even zero correlation) is a goodexample (high entropy).

The coefficient K can be the same for each mixer as in the preferredembodiment or different for all the mixers.

The processing carried out in the device of FIG. 1 is applied only tothe luminance component of the video.

In practice, a processing of the chrominance component may provokedisagreeable artefacts in colour, and, above all, it is the luminancecomponent that has most of the complexity of the image.

The signal E_(in) (identical to the signal E) at the output of themodule 2 is then subjected to erosion in the module 3. The erosionprocess consists in keeping the pixel that has the minimum luminancevalue among the pixels of a structuring element that it receives asinput. The structuring element comprises a 3*3 window, three rows bythree columns, around the current pixel, or 9 pixels. However, anotherwindow size can be found bearing in mind that the size of the window isdirectly proportional to the severity of the erosion.

The module 3 therefore computes for each pixel of the incoming videosignal E_(in), its new luminance value.

The video signal T0 at the output of the module 3 therefore representsthe eroded signal E_(in), or therefore for each pixel, its luminancevalue modified in relation to T0 corresponding to the minimum value ofthe structuring element of which it is part.

Then, the signal T0 is transmitted to a mixer 4. The mixer 4 alsoreceives as input the coefficient K transmitted by the module 2.

The mixer 4 produces as output the signal S0 according to the followingformula:

s0=K×T0+(1−K)×Ein

The signal S0 is input into the dilatation module 5.

The dilatation operation consists in retaining the pixel that has themaximum luminance value among the elements of a structuring elementcentred on the current pixel, with a size of 3×3=9 pixels.

The dilatation module produces as output a new video signal T1transmitted to the input of a mixer 6. The mixer 6 also receives asinput the weighting coefficient K.

The mixer 6 then produces a signal S1 according to the following formulaby weighting, for each pixel, the luminance value that it receives asinput:

S1=K×T1+(1−K)×S0

The signal S1 is then input into a second dilatation module 7.

The dilatation module 7 then performs an dilatation operation on thesignal S1. The dilatation operation consists in retaining the pixel thathas the maximum luminance value among the elements of a structuringelement centred on the current pixel with a size of 3×3=9 pixels.

The module 7 then produces as output a signal T2 which is input into themixer 8 which weights the luminance component of the signal received,for each pixel received. The mixer 8 also receives as input theweighting coefficient K.

The mixer 8 produces as output a signal S2, according to the formula

S2=K×T2+(1−K)×S1

The signal S2 is then input into a second erosion module 9.

The module 9 applies erosion to the signal S2, the erosion operationconsisting as indicated previously in replacing the current pixel withthe minimum pixel among the pixels of the structuring element defined aspreviously.

The erosion module 9 produces as output a signal T3 which is input intoa fourth mixer 10. The mixer 10 also receives as input the weightingcoefficient K.

The mixer produces as output a signal S3 according to the followingformula:

S3=K×T3 +(1−K)×S2

The signal S3 is then transmitted to the input of an interlacer 11 whichis used to interlace the signal S3 in order to obtain the video outputSi of the pre-processing device which is then transmitted to an encodingdevice.

The incoming video signal Si then benefits from a reduced spatialentropy and its subsequent encoding is made easier. Any encoding typecan be considered subsequently.

Naturally, the invention is not limited to the embodiment describedabove. A person skilled in the art will easily understand that modifyingthe number of morphological operations of the pre-processing device canbe considered, as can modifying the number of associated mixers.

An erosion followed by a dilation is called an opening.

A dilatation followed by an erosion is called a closing.

1-10. (canceled)
 11. Method of processing an image of a video imagesequence, wherein it comprises the following successive steps: a stepfor computing a complexity value representative of the complexity ofsaid image; a first step of morphological processing applied on saidimage, said first step generating a first processed image; a second stepfor mixing said image and said first processed image depending on saidcomplexity value, said second step generating a mixed image; a thirdstep of morphological processing applied on said mixed image, said thirdstep generating a second processed image; and a fourth step for mixingsaid mixed image and said second processed image depending on saidcomplexity value.
 12. Method according to claim 11, wherein saidcomplexity value is the intra-image correlation.
 13. Method according toclaim 11, wherein said second step consists in a linear combination ofsaid image and said first processed image, said combination depending ona first weighting coefficient computed from said complexity value. 14.Method according to claim 13, wherein said fourth step consists in alinear combination of said mixed image and said second processed image,said combination depending on a second weighting coefficient computedfrom said complexity value.
 15. Method according to claims 14, whereinsaid first and second weighting coefficients are equal.
 16. Methodaccording to claim 15, wherein said first and second weightingcoefficients are proportional to said complexity value.
 17. Methodaccording to claim 11, wherein said first step is an erosion.
 18. Methodaccording to claim 11, wherein said third step is a dilatation. 19.Device for processing an image of a video image sequence, wherein itcomprises: means for computing a complexity value representative of thecomplexity of said image; means for applying a morphological processingon said image, said means generating a first processed image; means formixing said image and said first processed image depending on saidcomplexity value, said means generating a mixed image; means forapplying a morphological processing on said mixed image, said meansgenerating a second processed image; and means for mixing said mixedimage and said second processed image depending on said complexityvalue.